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ABSTRACT 


Both the Institutional Functioning Inventory (IFI) 



and the College and University Scales (CUES) are self-study and 
evaluation instruments* Because the IFI is concerned primarily with 
dimensions of interest to faculty, while CUES reflects aspects of 
campus life mere important to students, these two measures are 
frequently used together to obtain a complete perspective* It may be 
disconcerting when the user finds that each employs a different 
scoring technique. The IFI is scored in the typical measurement 
fashion where each item answered in the keyed direction is scored "1 M 
and those in a non-keyed direction are scored "O.” The scoring system 
for CUES is quite different; it utilizes the n 66+/33- M method, a 
technique which takes into account only those items about which there 
is a consensus of two-to-one or greater among the respondents* IFI 
data from the 37 institutions comprising the IFI normative group was 
rescored using the CUES method* The comparison of the two scoring 
techniques indicated that the two methods are practically identical 
in terms of the resulting institutional profiles. However, a variety 
of reasons are given for the retention of the traditional scoring 
method with IFI* (CK) 
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A NOTE ON THE COMPARABILITY OF ALTERNATIVE SCORING METHODS 
FOR THE INSTITUTIONAL FUNCTIONING INVENTORY 

Rodney T. Hartnett 

The Institutional Functioning Inventory (iFl) is an institutional self- 
study and evaluation instrument by which colleges or universities can describe 
themselves in terms of various dimensions judged to be important and relevant 
for the well-being of institutions of higher education. Certain members of the 
college community --usually members of the faculty--are asked to report their 
perceptions of various emphases and characteristics which seem to highlight 
their institutional environment.^ 

A similar purpose is served by the College and University Environment 

Scales (CUES) , an instrument which has proved useful for describing college 

o 

environments since its publication in 1963- CUES, however, was designed 
principally for use with college students . Furthermore, these two instruments 
differ not only in that perceptions are obtained from different reporters, 
but also in the kinds of institutional variables and emphases that are tapped. 
Though both instruments measure some campus characteristics of general interest 
to both students and faculty, the scales in CUES quite naturally tend to re- 
flect dimensions of campus life that are more Important to students, such as 
Community (is it a friendly, cohesive campus?) and Propriety (is the campus 
atmosphere proper and conventional?), whereas the scales in the IF I tend to 

“^Peterson, R. E Centra, J. A., Hartnett, R. T . , and Linn, R. L. 
Institutional Functioning Inventory: Preliminary Technical Manual . Princeton, 

N. J*: Educational Testing Service, 1970. 

2 

Pace, C. R* College & University Environment Scales (CUES) Second Edition 
Technical Manual. Princeton, N. J.: Educational Testing Service, 1969 . 
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reflect dimensions more important to members of the faculty , such as Freedom 
(academic freedom) and Concern for Advancing Knowledge (the degree to which 
the institution emphasizes research) . 

Nevertheless, both instruments appear to be getting at somewhat different 
aspects of the same general question: "What is this place like?" Consequently 

they can and have been used conjointly by institutions seeking to gain a com- 
plete picture of how their institution is perceived by various relevant members 
of the c ampu s c ommu nity . 

Given this situation, users of these two measures of college environments 

may find it somewhat disconcerting that each uses a different method of scoring. 

The IF I is scored in the typical measurement fashion, where each item answered 

in the keyed directicn is scored !! 1 !! and those in a non-keyed direction are 

scored ,f 0. ,r Each respondent's score, therefore, is the number of items answered 

3 

in the keyed direction. Scores of individuals are then averaged to give an 
institutional score on the scale in question. 

The scoring system for CUES, however, is quite different, at least in 
appearance. It is scored by means of the n 66 + /33- t! method, a technique which 
takes into account only those items about which there is a consensus of two- 
to-one or greater among the respondents (students). The l, 66 + /33- M method con- 
sists of adding the number of items answered by 66 percent or more of the 
students in the keyed direction, subtracting the number of items answered by 
33 percent or fewer of the students in the keyed direction, and adding a 
constant to the difference in order to eliminate negative scores. The rationale 
for such a procedure, as explained in the CUES manual, is as follows: 



% 



or details regarding IF I scoring, see Peterson, et al., op. cit . 
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Firstj we regard CUES as an opinion poll. The percent of 
people agreeing or disagreeing with a statement is the commonly 
accepted manner of reporting opinion poll results . Second, we 
are interested only in what is judged to be characteristic of 
the environment, and therefore we have to decide how much agree- 
ment there needs to be in order to justify calling something 
characteristic. If half the students agree and half disagree, 
then obviously the result cannot be described as characteristic 
because we define the word characteristic to mean dominant, 
not average. Third, continuing this rationale, the score for 
a scale is determined by the number of statements that have 
been judged as characteristic of the environment, with char- 
acteristic defined as a level of consensus at least two-to-one 
or greater.^ 

Such reasoning, if defensible for CUES, is equally compelling for the IFI. 
If it is true that the M 66 + /33“ M scoring method (hereafter called the CUES 
method) is a better procedure for reporting institutional scores, then such a 
technique should be employed with the IFI. But does it make any difference? 
Though the arguments made for the CUES method are conceptually sound, the 
question remains: Would such a scoring procedure actually result in a different 

rank- ordering of institutions than one obtained from the currently employed 
IFI scoring method? 



The Findings 

IFI data from the 37 institutions comprising the IFI normative group were 
rescored using the CUES method. Specifically, for each institution all items 
answered in the keyed direction by 33 percent or fewer of the faculty members 
were subtracted from the total number of items answered by 66 percent or more 
in the keyed direction, and a constant of 12 (which is the number of items in 
each IFI scale) was added to this difference- Thus, scores obtained by the 
CUES method could theoretically range from 0 to 2h, whereas scores obtained in 
the "regular" fashion could theoretically range from 0 to 12. 

^ ^Pace, op. cit . , p. lb. 

ERIC 
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The 37 IF I means calculated by the CUES method were then correlated with 
the means obtained via the regular procedure. Those correlations are presented 
in Table 1 . 

It is clear that the rank-ordering of the 37 institutions on any of the 
scales is barely affected by the different scoring procedure. The lowest 
correlation between the two sets of institutional means is . 9h > which occurs 
for two IF I scales. Seven of the 11 correlations are .97 or higher.^ Further- 
more , the interrelationships among the 11 scales are only minimally influenced 
by the CUES scoring procedure. As indicated in Table 2, the intercorrelations 
among the scales scored by the traditional method are strikingly similar to 
the correlations among the scales scored by the CUES method. The largest dif- 
ference occurs for the HD/lS correlations, where the difference between the 
two squared correlations (rounded) is .11* ( . 71 - .6<J ) . 

Conclusions 

A comparison of the scoring techniques — the traditional psychometric 
method and the CUES method~~as applied to the Institutional Functioning Inven- 
tory, indicates that the two methods are practically identical in terms of 
the resulting institutional profiles. Thus, in terms of an institutions 
concern about its mean scale scores relative to other institutions, the choice 
of scoring procedure is not important. However, in view cf the nearly identical 
results, there are other reasons for preferring the traditional psychometric 
technique . 

^It is perhaps worth noting that these correlations are similar in magni- 
tude to CUES intercorrelations reported by Pace in the preliminary technical 
manual, where the two scoring techniques employed were the traditional psycho- 
metric method and a method similar but not identical to the ,l 66 + /33- M technique 
described above. For the five CUES scales the correlations ranged from .88 
to .98. See Pace’s College and University Environment Scales: Preliminary 

Technical Manual . Educational Testing Service, 19&3. 

O 
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Table 1 

Correlations between IF I Scale Means 
Obtained from Alternative Scoring Procedures 

Correlation between 



Scale Institutional Means 

Intellectual -Aesthetic Extracurriculum (IAE) .98 

Freedom (F) -97 

Human Diversity (HD) -97 

Concern for Improvement of Society (IS) -98 

Concern for Undergraduate Learning (UL) -96 

Democratic Governance (EG) *97 

Meeting Local Needs (MLN) *97 

Self-Study and Planning (SP) .9h 

Concern for Advancing Knowledge (AK) *99 

Concern for Innovation ( Gl) *96 

Institutional Esprit (IE) *9h 
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Table 2 



IFI Scale Intercorrelations Using 
Alternative Scoring Methods 



Intercorrelations Based on CUES Scoring Method 
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Decimals omitted from correlations. See Table 1 for complete names of 
scales . 
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First, the traditional scoring technique enables users of the data to 
easily obtain other useful, descriptive statistical measures not possible with 
the CUES technique. Measures of the distribution of scale scores (e.g., average 
deviation, standard deviation, variance), for example, provide illuminating data 
about the variety of perceptions held by members of the observer group (e.g., 
faculty, students) and serve as another gauge of consensus among the reporters 
on the total scale. (The IFI manual, it should be noted, provides a norm-group 
distribution of institutional standard deviations for each scale to enable users 
to compare the spread of scores at their institution with those at other col- 
leges.) Furthermore, information about reporter agreement on each item (the 
basis for the CUES technique) is still provided with the item analysis informa- 
tion that is part of the routine score reporting service for the IFI. 

Second, there is the simple fact that the CUES method may be unfamiliar, 
perhaps even suspect, to many who will be interpreting data gathered by means 
of the IFI. Since most research and measurement people (including institutional 
researchers) are accustomed to thinking about test scores and inventory results 
in terms of the typical measures of central tendency and their accompanying 
indicators of the variance or spread of scores, it T s quite possible that the 
CUES method of scoring might result in score misinterpretation. Hence, for the 
convenience of the users of the IFI, the traditional scoring method would seem 
to be preferable. 

Finally, the traditional scoring technique provides the research oriented 
user of the IFI with a wider array of methodological tools. For example, cor- 
relations between various individual respondent characteristics (e.g., teaching 
load, professorial rank) and IFI scales can be computed with scores obtained in 
the traditional way but not for scores resulting from the CUES method. 



